Method and apparatus for capturing images

ABSTRACT

A first set of image data is acquired based on a first frame of light entering the camera before receiving an instruction to capture an image from a user. The first set of image data is then used to test the performance of at least one hardware component in the camera.

REFERENCE TO RELATED APPLICATIONS

The present application is a Continuation of U.S. patent applicationSer. No. 09/505,223, filed on Feb. 16, 2000, which claims prioritybenefit from U.S. Provisional Application 60/136,517, filed on May 28,1999 and entitled “EARLY IMAGE ACQUISITION”.

BACKGROUND OF THE INVENTION

The present application relates to digital cameras. In particular, thepresent invention relates to image control and processing in digitalcameras.

Digital cameras capture images by converting light into electricalsignals and processing the electrical signals to produce a set of imagedata. The image data is then stored in a long-term memory for laterretrieval.

The processing of captured images is compute-intensive, often making theuser wait for the final picture after the shutter button is pressed. Forusers who want to take a series of photographs in rapid succession, thisdelay is undesirable.

SUMMARY OF THE INVENTION

A first set of image data is acquired based on a first frame of lightentering the camera before receiving an instruction to capture an imagefrom a user. The first set of image data is then used to test theperformance of at least one hardware component in the camera.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a general block diagram of a camera under an embodiment of thepresent invention.

FIG. 2 is a flow diagram of a method under one embodiment of the presentinvention.

FIG. 3 is a block diagram of software components under one embodiment ofthe present invention.

FIG. 4 is a flow diagram of an alternative method under one embodimentof the present invention.

FIG. 5 is a flow diagram of an alternative method under one embodimentof the present invention.

FIG. 6 is a flow diagram of a method of verifying pre-processing resultsunder one embodiment of the present invention.

DETAILED DESCRIPTION OF ILLUSTRATIVE EMBODIMENTS

FIG. 1 is a block diagram of a camera 200 under an embodiment of thepresent invention. Light 202 enters camera 200 through a lens assembly204, which focuses the light onto a photo array 206. In manyembodiments, lens assembly 204 is controlled by an auto-focus controller208 that modifies lens assembly 204 to bring an image into focus onphoto array 206.

Photo array 206 is constructed of an array of specialized transistorsknown as Charge Coupled Devices (CCDs). For black-and-white images, eachCCD represents a separate pixel in the captured image. For color images,a set of CCD pixels with overlaid color filters are combined torepresent a single pixel, with each CCD providing information on adifferent color of light.

Based on the output of the CCDs, photo array 206 transmits an analogsignal to an analog-to-digital (A/D) converter 210. A/D converter 210converts the analog signal into a series of digital values, with eachvalue representing the brightness or intensity of a pixel's color. Thedigital values produced by A/D converter 210 are stored in a register212 that receives read and write control signals from an image processor214. By controlling when A/D converter 210 can write to register 212,image processor 214 is able to capture and store values representing asingle frame of light.

Image processor 214 and auto-focus controller 208 are able to access anduse the image data in register 212. Auto-focus controller 208 uses thedata to adjust lens assembly 204. Image processor 214 uses the imagedata for a number of processing functions described further below.

In most embodiments, image processor 214 processes the image data inregister 212 using software components stored in code storage 216.Intermediate results of this processing are stored in a second register218, and the final image data resulting from the processing is stored inlong-term storage 220.

In most embodiments, image processor 214 receives input from a capture.button 222 that is depressed by the user to indicate when the user wantsto capture an image. In some embodiments, the user is able to presscapture button 222 half-way down to indicate that they want the camerato prepare to capture an image by, for example, activating theauto-focus feature of the camera.

Image processor 214 is also able to set parameters for a flash 224,which is controlled by a flash controller 226. The flash parameters arestored in register 228 by image processor 214 and include parameterssuch as a red-eye reduction flash parameter, a duration parameter, and abrightness parameter.

Under the present invention, image processor 214 performs someprocessing functions before the user indicates that they wish to capturean image. Thus, before capture button 222 is fully depressed, imageprocessor 214 performs one or more functions such as white balance,contrast adjustment, and red-eye reduction. After the user fully depresscapture button 222, image processor 214 performs additional post-captureprocessing functions. The results of the pre-capture and post-captureprocessing functions are then combined to produce a final set of imagedata. By performing some of the image processing functions before theuser tries to capture an image, the present invention reduces thepost-capture processing time.

FIG. 2 is a flow diagram of a method for capturing images under oneembodiment of the present invention. Under one embodiment, the processof FIG. 2 is performed by image processor 214 by invoking a set ofsoftware components. FIG. 3 provides a block diagram of some of thesecomponents, which are described below in connection with the method ofFIG. 2.

The method of FIG. 2 begins at step 300 and proceeds to step 302 where asystem control component 400 invoked by image processor 214 causes imagedata for a frame of light to be written to register 212. System control400 then retrieves the stored values from register 212 and at step 304invokes a pre-capture process control 402 that controls one or morepre-capture processing functions.

Under one embodiment, pre-capture process control 402 invokes one ormore separate image processing components such as white balancecomponent 404, red-eye reduction component 406, contrast adjustmentcomponent 408, flesh tone correction component 410, and edge enhancementcomponent 412 of FIG. 3. When invoked, each of these components is givenaccess to the image data. When a component completes its operation, ittypically returns one or more processing values to pre-capture processcontrol 402. Under some embodiments, pre-capture process control 402invokes two or more of the image processing components in parallel. Inother embodiments, the image processing components are invoked serially.

In some embodiments, pre-capture process control 402 also invokesprocessing functions to test the performance of some hardware componentsof the camera. For example, under one embodiment, pre-capture processcontrol 402 invokes a defective pixel detection component 428, whichdetermines if one or more of the CCD transistors is malfunctioning. Inother embodiments, pre-capture process control 402 invokes a memoryavailability component 430, which determines if long term image storage220 of FIG. 1 has enough available memory to accommodate the next image.If there is insufficient available memory, system control 400 eithersends a message to the user through a display driver 423 in operatingsystem 416 or reallocates the memory resources by remapping virtualaddress space, terminating other components or minimizing othercomponents. When system control 400 reallocates the memory resources, itacts as a memory management component. As shown in FIG. 1, this messageis provided to the user through a display 230 that is controlled by adisplay interface 232. Display interface 232 receives its display datafrom display driver 423 in image processor 214.

Returning to the method of FIG. 2, after step 304, system control 400checks to see if the user has pressed the image capture button at step306. Under some embodiments, information as to whether the user haspressed the button is provided by a user input component 414 shown inoperating system 416 in FIG. 3. If the capture button has not beenpressed at step 306, the process of FIG. 2 returns to step 302, andsteps 302 and 304 are repeated.

If the capture button has been pressed at step 306, system controlcomponent 400 captures image data for a second frame of light at step308. System control component 400 then invokes a post-capture processcontrol component 418 that invokes one or more image processingcomponents at step 310. Some of the image processing components, such ascompression component 420 of FIG. 3, change the image data as theyprocess it. If another image processing component is invoked after acomponent that changes the image data, the subsequent image processingcomponent receives the modified image data. Post-capture process control418 also acts as an image production control that utilizes the resultsof the pre-capture processing components and the post-capture processingcomponents to produce a final set of image data.

At step 312, the final set of image data is stored in long-term imagestorage 220 of FIG. 1. Under the embodiment of FIG. 3, the process ofstoring the final image data is controlled by operating system 416.After the final image data has been stored, the process of FIG. 2 endsat step 314.

FIG. 4 provides a flow diagram for an alternative method of capturingimages under the present invention. The process of FIG. 4 starts at step480 and continues at step 482 where system control 400 waits for theuser to depress the capture button. When the user depresses the capturebutton, system control 400 uses a flash control component 422 inoperating system 416 of FIG. 3 to trigger a red-eye reduction flash atstep 484 of FIG. 4. At step 486, system control 400 captures image datagenerated from a frame of light that includes light from the red-eyereduction flash. The red-eye reduction flash illuminates the subject ofthe photograph so that the captured image has color, contrast, andbrightness characteristics more similar to the image produced when themain flash is later triggered, as described below.

At step 488, system control 400 invokes pre-capture process control 402,which invokes one or more processing components as described above.

In addition to using some or all of the processing components describedabove, some embodiments that utilize the method of FIG. 4 perform anadditional pre-capture processing function. This additional function isperformed by flash control 424 in FIG. 3 and involves adjusting theparameters of the camera's flash system. For example, in someembodiments, flash control 424 changes the duration and/or brightness ofthe flash. The changes to the flash parameters are stored in register228 of FIG. 1 through flash control 422 of operating system 416.

When the pre-capture processing is complete, the method of FIG. 4continues at step 490 where system control 400 triggers the main flash.The image generated by the light of the main flash is then captured atstep 492. At step 494, system control 400 invokes post-capture processcontrol 418, which invokes one or more post-capture processing functionsas described above. Post-capture process control 418 utilizes theresults of the pre-capture processing and post-capture processing toform a final set of image data that is stored before the process of FIG.4 ends at step 496.

FIG. 5 provides a flow diagram of an alternative method for capturingimages under the present invention. The method of FIG. 5 begins at step520 and proceeds to step 522 where system control 400 waits to receive apre-capture event. The pre-capture event can be generated by a softwareroutine based on a timer or some other triggering event or can begenerated when the user presses the capture button half-way down. Whenthe pre-capture event occurs, the method of FIG. 5 continues at step 524where system control 400 captures image data for a first frame of light.At step 526, system control 400 invokes pre-capture process control 402,which invokes one or more pre-capture processing components as describedabove in connection with FIGS. 2 and 3. In the method of FIG. 5, thepre-capture processing components can include an auto-focus componentsuch as auto-focus component 426 of FIG. 3.

After pre-capture process control 402 has finished executing, systemcontrol 400 waits for the user to fully depress the capture button atstep 528. When the user fully depresses the capture button, systemcontrol 400 retrieves a new set of image data based on a second frame oflight that entered the camera just after the capture button wasdepressed. This is shown in step 530 of FIG. 5. System control 400 theninvokes post-capture process control 418 at step 532. Post-captureprocess control 418 invokes one or more post-capture processingcomponents and utilizes the results from the pre-capture processingcomponents and the post-capture processing components to form a set offinal image data. System control 400 then stores the final image databefore the method of FIG. 5 ends at step 534.

Under some embodiments of the invention, post-capture process control418 invokes a verify pre-capture results component 432 to verify theresults from one or more of the pre-capture components before utilizingthose results. Such verification is helpful because the lightingconditions may have changed between the time when the image data for thefirst frame of light was collected and the time when the image data forthe second frame of light was collected. FIG. 6 is a flow diagram of onemethod for verifying the pre-capture processing results.

The method of FIG. 6 begins at step 600 where a portion of the imagedata for the second frame of light is selected. At step 602, thissub-set of image data is provided to the pre-capture processingcomponent that is having its results verified. This produces averification result that is compared to the original result produced bythe pre-capture processing component for the first frame of light. Thiscomparison is performed in step 604. The difference between the tworesults is compared to a threshold at step 606. If the two resultsdiffer by more than the threshold, all of the image data for the secondframe of light is passed to the pre-capture processing component togenerate a new result at step 608. If the two results differ by lessthan the threshold, the original result formed from the first frame oflight is used at step 610. Thus, if image conditions have changedsubstantially between the first and second frames of light, thepre-capture processing functions are repeated using the second frame oflight instead of the first frame of light. However, if image conditionshave not changed substantially between the frames of light, the resultsformed from the first frame of light are used to determine the finalimage data.

Although the present invention has been described with reference toparticular embodiments, workers skilled in the art will recognize thatchanges may be made in form and detail without departing from the spiritand scope of the invention.

1. A method in a camera, the method comprising: acquiring a first set ofimage data based on a first frame of light entering the camera beforereceiving an instruction to capture an image from a user; and testingthe performance of at least one hardware component in the camera usingthe first set of image data.
 2. The method of claim 1 wherein testingthe performance of at least one hardware component comprises determiningif the camera has enough available memory to store the first set ofimage data.
 3. The method of claim 2 further comprising determining thatthe camera does not have enough available memory to store the first setof image data and indicating to the user that there is insufficientmemory before acquiring a second set of image data.
 4. The method ofclaim 2 further comprising determining that the camera does not haveenough available memory to store the first set of image data andreallocating memory before acquiring the second set of image data. 5.The method of claim 4 wherein reallocating memory comprises terminatinga software component.
 6. The method of claim 4 wherein reallocatingmemory comprises minimizing a software component.
 7. The method of claim4 wherein reallocating memory comprises remapping virtual address space.8. The method of claim 1 wherein testing the performance of at least onehardware component comprises determining if a light sensor is working.9. The method of claim 8 wherein testing the performance of at least onehardware component comprises determining if each light sensor in anarray of light sensors is working.
 10. The method of claim 9 furthercomprising providing an indication of which light sensors are notworking before acquiring a second set of image data.
 11. The method ofclaim 1 further comprising receiving a pre-capture event beforeacquiring the first set of image data.
 12. The method of claim 11wherein the pre-capture event is an indication that a user has pressed acapture button half-way down.
 13. The method of claim 11 wherein thepre-capture event is produced by a software routine.
 14. A camera havingprocessor-executable components for capturing images, the componentscomprising: an image acquisition component capable of acquiring imagedata representing a first frame of light; and a hardware testingcomponent capable of testing a hardware component in the camera based onimage data acquired by the image acquisition component for the firstframe of light.
 15. The camera of claim 14 wherein the hardware testingcomponent comprises a memory availability component capable ofperforming a memory resource function to determine if there issufficient available memory to store the image data.
 16. The camera ofclaim 15 further comprising a user notification component capable ofnotifying the user when there is insufficient memory to store the imagedata before the image acquisition component acquires image data for asecond frame of light.
 17. The camera of claim 15 further comprising amemory management component that is capable of reallocating memoryresources before the image acquisition component acquires image data fora second frame of light.
 18. The camera of claim 17 wherein the memorymanagement component reallocates memory resources by terminatingcomponents.
 19. The camera of claim 17 wherein the memory managementcomponent reallocates memory by remapping a virtual address space. 20.The camera of claim 14 wherein the hardware testing component comprisesa defective pixel detection component that is capable of identifyingdefective light sensors in the camera based on image data acquired forthe first frame of light.